Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@AI/kaggle_notebooks/itransformer/notebook.ipynb`:
- Around line 73-90: The current logic may load and run train_kaggle.py twice
and attempts a fallback using a non-existent path; replace the exec/open(...)
path with a single importlib-based loader: check that train_script exists and if
not raise/print and exit immediately, otherwise use
importlib.util.spec_from_file_location('train_kaggle', train_script),
module_from_spec(spec), spec.loader.exec_module(module) and call module.train();
remove the exec(open(...).read()) and the erroneous else branch that reuses the
missing path so training is loaded and invoked exactly once via module.train().

In `@AI/kaggle_notebooks/patchtst/notebook.ipynb`:
- Around line 36-41: Remove the second and third notebook cells that copy/load
non-existent files and keep the first cell's clone + sys.path approach; instead
import PatchTST_Model and the feature functions directly from the cloned repo
(from AI.modules.signal.models.PatchTST.architecture import PatchTST_Model and
from AI.modules.features.legacy.technical_features import
add_technical_indicators, add_multi_timeframe_features), set any needed env vars
(e.g., PARQUET_DIR, WEIGHTS_DIR), and if you must run the training script
dynamically, load the repository's train_kaggle.py via
importlib.util.spec_from_file_location pointing to
/kaggle/working/sisc-web/AI/modules/signal/models/PatchTST/train_kaggle.py and
invoke its train() entry point instead of copying non-existent files; also
ensure kernel_sources in kernel-metadata.json is populated or left empty
consistently with this import strategy.

In `@AI/kaggle_notebooks/TCN/notebook.ipynb`:
- Around line 69-79: Replace the exec-based loading of train_script with an
importlib-based dynamic module import: when train_script exists, create a module
spec from that path (use importlib.util.spec_from_file_location and
importlib.util.module_from_spec), load the module via spec.loader.exec_module,
then call the exported train_model_kaggle function from the loaded module (guard
the call with hasattr/module attribute check and raise or log a clear error if
train_model_kaggle is missing), and remove the open(...).read() pattern so no
file descriptor is left open; keep the existing fallback print for the else
branch.

In `@AI/kaggle_notebooks/transformer/notebook.ipynb`:
- Around line 48-51: The code sets os.environ before importing or defining os
which causes a NameError; add an import os at the top of the cell (before any
use of os.environ) so the os module is available; locate the cell containing the
os.environ assignments (references to os.environ and the symbol os) and insert
import os as the first statement in that cell.
- Around line 36-39: 이 셀에서 발생하는 NameError/ModuleNotFoundError를 고치려면 먼저 shutil를
import하고 기술 지표 모듈을 올바른 경로로 가져오세요: 현재 코드의 from technical_features import
add_technical_indicators, add_multi_timeframe_features 대신 이미 사용중인
AI.modules.features.legacy.technical_features에서 add_technical_indicators와
add_multi_timeframe_features를 import하거나 기존 import(예:
AI.modules.features.legacy.technical_features)를 재사용하도록 수정하여 모듈 경로를 정확히 참조하고
shutil.copy 호출 전에 shutil이 import되었는지 확인하세요.

In `@AI/modules/signal/core/artifact_paths.py`:
- Around line 104-110: The resolver is returning lowercase "patchtst" paths
while training/saving uses "PatchTST", causing mismatched directories on
case-sensitive filesystems; update the handling in
AI/modules/signal/core/artifact_paths.py so that when normalized_model ==
"patchtst" you resolve to the same cased directory and filenames used by the
trainer (use "PatchTST" for resolved_model_dir and the matching artifact
filenames), i.e. change the resolved_model_dir, model_path and scaler_path
construction in the ModelArtifactPaths return so they reference "PatchTST" and
the exact saved filenames produced by AI/modules/signal/models/PatchTST/train.py
(use the same casing and basename as the saver).

In `@AI/modules/signal/models/itransformer/train_kaggle.py`:
- Around line 160-171: The scaler is being refit per ticker inside the for
ticker, group in df.groupby("ticker") loop (using scaler.fit_transform when
fit_scaler=True), causing leakage and inconsistent statistics; instead, if
fit_scaler is True, call scaler.fit once on the full dataset (e.g., on
df[available_feats].values.astype(np.float32)) before entering the loop, then
inside the loop only call scaler.transform on feat_vals; if fit_scaler is False,
ensure scaler is already fitted and continue using scaler.transform for
feat_vals (reference symbols: df.groupby("ticker"), available_feats, feat_vals,
fit_scaler, scaler.fit_transform, scaler.transform).

In `@AI/modules/signal/models/TCN/tcn_train_kaggle.py`:
- Around line 201-203: best_state is currently assigned a shallow reference to
model.state_dict(), so it gets mutated during later training and ends up
containing the final weights; replace assignments that save the best model
(where best_state = model.state_dict()) with a deep copy: import copy and set
best_state = copy.deepcopy(model.state_dict()) so best_state holds an
independent snapshot of parameters at the best validation loss.

In `@AI/modules/signal/models/TCN/wrapper.py`:
- Around line 217-233: The batch-prep loop uses get_standard_training_data()
directly which bypasses the same preprocessing used by predict(); update the
loop over ticker_data_map in the TCN wrapper to call
_prepare_dataframe(df.copy()) (or the same helper used by predict()) to perform
required column validation and inf/NaN cleaning before extracting
self.feature_columns, then proceed with seq_len checks, scaling
(self.scaler.transform) and appending to tensor_list and valid_tickers; ensure
you still handle exceptions and preserve the existing behavior around seq_len,
feature_columns, scaler, tensor_list and valid_tickers so batch and single-shot
prediction use identical preprocessing.

In `@AI/modules/signal/models/transformer/transformer_train_kaggle.py`:
- Around line 71-73: The saved artifact names in transformer_train_kaggle.py
('model_name' and 'scaler_name') use unsuffixed filenames which will not match
the resolver conventions in AI/modules/signal/core/artifact_paths.py (which
expects names like multi_horizon_model_prod.keras /
multi_horizon_scaler_prod.pkl or *_test for tests); update the 'model_name' and
'scaler_name' values to include the proper suffix (e.g., append _prod) so they
match the resolver lookup scheme and ensure the stored files can be found during
download/deploy.
- Around line 149-167: The current shuffle-based train_test_split after calling
loader.create_dataset causes temporal leakage; instead, perform a chronological
split on full_df before creating sequences so the scaler is fit only on training
data and validation windows are strictly later. Concretely: split full_df by a
date cutoff into train_df and val_df (no random shuffle), call
loader.create_dataset(train_df, ...) to produce X_ts_train, X_ticker_train,
X_sector_train, y_train and fit the scaler there, then call
loader.create_dataset(val_df, ..., scaler=trained_scaler or with a flag to only
transform) to produce X_ts_val, X_tick_val, X_sec_val, y_val; remove the later
train_test_split(...) shuffle=True usage and ensure horizons / n_outputs logic
remains unchanged.

In `@AI/pipelines/weekly_routine.py`:
- Around line 155-172: The script currently ignores run_weekly_pipeline()'s
result so the process always exits 0; capture its return or exception in the
__main__ block and propagate failure by exiting with a non-zero code: call
result = run_weekly_pipeline(...), then if result indicates failure (e.g., False
or a non-zero status) call sys.exit(1) or re-raise the caught exception;
alternatively wrap run_weekly_pipeline(...) in try/except and on exception log
the error and call sys.exit(1). Ensure you import sys if needed and reference
the existing run_weekly_pipeline call in the __main__ block.

In `@AI/scripts/deploy_to_server.py`:
- Around line 89-112: The deploy_model function currently always returns True
even if files are missing or transfer errors occur; update deploy_model to track
failures by creating a failed list/local flag: for each fname in model['files']
check os.path.exists(local_path) and append fname to failed if missing, wrap
scp.put(local_path, remote_path) in a try/except to append fname on exception,
and at the end return False if failed is non-empty (and optionally log which
filenames failed) otherwise return True; refer to deploy_model, model['files'],
local_path, remote_path, os.path.exists, scp.put and the printed messages when
implementing.
- Around line 73-74: Replace the unsafe AutoAddPolicy usage by enforcing host
key verification: instead of
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()), set
ssh.set_missing_host_key_policy(paramiko.RejectPolicy()) and load the known host
key from a secret (e.g. SSH_HOST_KEY) into the client before connecting; read
SSH_HOST_KEY from the environment/CI secret, parse it and add it with
ssh.get_host_keys().add(hostname, key_type, key_data) (or use
SSHClient.load_host_keys when writing the secret to a temp known_hosts file) so
the code around SSHClient() and set_missing_host_key_policy(...) performs strict
verification.

In `@AI/scripts/download_weights.py`:
- Around line 30-40: The download script currently can return success without
validating downloaded files, risking stale weights being reused; update the
logic in download_weights.py (referencing MODELS) to download each model into a
temporary directory first, verify presence and exact names of the expected files
(the same filenames deploy_to_server.py expects), only after validation
atomically replace/rsync into the model's dst_dir (empty/backup the dst_dir
before move), and fail/return False if any expected file is missing or names
differ; apply the same change to the code block around lines 61-80 that handles
additional models.

In `@AI/scripts/extract_to_parquet.py`:
- Around line 178-184: The SELECT in read_sql_safe currently omits two features
used downstream (correlation_spike and surprise_cpi), causing a mismatch with
AI/modules/signal/models/itransformer/train_kaggle.py; fix by adding
correlation_spike and surprise_cpi to the SELECT projection from
macroeconomic_indicators in extract_to_parquet.py (ensure read_sql_safe and
resulting DataFrame include these columns so they persist into the output
parquet).
- Around line 123-132: The retry loop leaks DB connections because conn.close()
is only called on success; update the loop around get_conn() and
pd.read_sql(query, conn) (inside the retry in extract_to_parquet.py) to always
close the connection on each attempt — either by using a try/finally that calls
conn.close() if conn is not None or by acquiring the connection with a context
manager — so every opened connection from get_conn() is closed even when
pd.read_sql throws.

In `@AI/scripts/trigger_training.py`:
- Around line 34-44: NOTEBOOKS entries use project_root joined to
"kaggle_notebooks/..." but the repo layout places notebooks under
AI/kaggle_notebooks, so update the NOTEBOOKS constant (the dicts for "PatchTST"
and "Transformer") to include the missing "AI" path segment when building
notebook_dir (e.g., include "AI" as an extra path component in os.path.join for
each notebook_dir) so trigger_notebook() can locate the folders.
- Around line 94-117: In the status-polling loop (the code that calls
subprocess.run(["kaggle", "kernels", "status", slug], ...) inside the
trigger_training flow), check result.returncode immediately after subprocess.run
and treat any non-zero return code as a hard failure: log/print the error
details (result.stderr or result.stdout), return False, and stop polling instead
of sleeping and retrying; keep the existing checks for
"complete"/"error"/"running" only when returncode == 0 to avoid unnecessary
5-minute retries (follow the same returncode handling pattern used in
trigger_notebook()).

In `@AI/scripts/upload_to_kaggle.py`:
- Around line 34-39: The subprocess.run call in upload_to_kaggle.py references
an undefined variable KAGGLE_DATA_DIR causing a NameError; change that argument
to use the existing OUTPUT_DIR (or rename OUTPUT_DIR to KAGGLE_DATA_DIR
consistently) so the call in subprocess.run([... "-p", KAGGLE_DATA_DIR, ...])
uses the defined upload directory; update any other occurrences to keep the
variable name consistent and ensure the kaggle upload command runs.

---

Minor comments:
In `@AI/kaggle_notebooks/TCN/notebook.ipynb`:
- Around line 60-62: The pip install cell currently runs "!pip install -q
scikit-learn pandas numpy torch" without pinned versions; update that cell (the
pip install command string) to pin package versions to reproducible releases
(e.g., ensure numpy is constrained to <2.0 as required by the
library_context/manifest and pin compatible versions of scikit-learn, pandas,
and torch) so the notebook installs deterministic, manifest-compatible packages.

In `@AI/scripts/deploy_to_server.py`:
- Around line 76-77: The current code only calls
paramiko.RSAKey.from_private_key(io.StringIO(SSH_KEY_STR)), which fails for
Ed25519/ECDSA keys; update the logic that creates private_key to try parsing
SSH_KEY_STR with multiple Paramiko key classes (e.g.,
paramiko.RSAKey.from_private_key, paramiko.ECDSAKey.from_private_key,
paramiko.Ed25519Key.from_private_key, and paramiko.DSSKey.from_private_key)
using io.StringIO(SSH_KEY_STR), catching exceptions and falling through until
one succeeds, and raise a clear error if none parse; reference the private_key
variable and the from_private_key calls to locate where to implement this
fallback parsing.

---

Nitpick comments:
In @.github/workflows/train.yml:
- Around line 58-63: Replace the fragile echo that constructs the Kaggle JSON
with a safe JSON builder: use jq (or printf piped to jq -n --arg) to create the
JSON object from the GitHub secrets and write it to ~/.kaggle/kaggle.json, then
set chmod 600; specifically update the "Setup Kaggle credentials" step to pass
${{ secrets.KAGGLE_USERNAME }} and ${{ secrets.KAGGLE_KEY }} as jq arguments
(instead of interpolating them into a raw string) to avoid JSON corruption or
shell injection.
- Line 31: Update the GitHub Actions steps that reference actions/checkout@v3
and actions/setup-python@v4 to their current stable releases: locate the `uses:
actions/checkout@v3` and `uses: actions/setup-python@v4` lines and bump them to
the latest recommended major versions (e.g., actions/checkout@v4 and
actions/setup-python@v5 or the most recent stable tags), then commit the change
and run a workflow lint/validation to ensure compatibility with any inputs you
rely on.

In @.gitignore:
- Line 47: 현재 .gitignore의 전역 패턴 "test_*.py"가 저장소 전체의 테스트 파일을 무시해 CI에 올라가지 못할 위험이
있습니다; .gitignore에서 "test_*.py" 항목을 제거하거나 범위를 좁혀 특정 디렉터리로 한정(예:
"scripts/test_*.py" 또는 "dev_tools/test_*.py" 등)하도록 수정하고, 변경한 패턴이 의도한 위치에서만 무시되는지
확인한 뒤 커밋하세요; 참조 대상: .gitignore의 "test_*.py" 패턴을 찾아 교체하거나 삭제하십시오.

In `@AI/kaggle_notebooks/TCN/notebook.ipynb`:
- Around line 33-42: The subprocess.run call that clones the repo doesn't check
for errors; update the clone logic that uses subprocess.run and the result
variable by either adding check=True to subprocess.run or explicitly inspecting
result.returncode (and using result.stderr/result.stdout) after the call; if
returncode != 0, log/print a clear error message including result.stderr and
raise an exception (e.g., RuntimeError or re-raise CalledProcessError) so
downstream cells don't proceed on a failed git clone.

In `@AI/modules/signal/models/itransformer/train_kaggle.py`:
- Around line 326-327: The dict entries for "n_train_samples" and
"n_val_samples" wrap len(...) with an unnecessary int() call; remove the int()
so use "n_train_samples": len(X_train) and "n_val_samples": len(X_val) (locate
these keys in the training metadata construction in train_kaggle.py).

In `@AI/modules/signal/models/PatchTST/train_kaggle.py`:
- Around line 185-194: The except block in the try around
add_technical_indicators and add_multi_timeframe_features swallows the original
exception; update the handler in train_kaggle.py to re-raise a RuntimeError
using "raise RuntimeError(... ) from e" so the exception chain is preserved and
include the original exception message, and consider narrowing the except to
specific exceptions thrown by
add_technical_indicators/add_multi_timeframe_features (e.g.,
ValueError/TypeError) instead of a bare Exception; keep the existing
fail_count/fail_limit logic and logging but ensure the re-raise uses "from e".
- Line 321: Replace the unnecessary f-string in the final print statement:
locate the print call in train_kaggle.py that currently uses print(f"\n>> 완료")
and change it to a normal string print (remove the leading "f") so it reads
print("\n>> 완료"), leaving the rest of the function and surrounding code
unchanged.

In `@AI/modules/signal/models/PatchTST/wrapper.py`:
- Around line 191-193: Update the get_signals method to match the
BaseSignalModel contract by adding the explicit ticker_id and sector_id
parameters (e.g., def get_signals(self, df, ticker_id=None, sector_id=None,
**kwargs)) while keeping the current behavior of delegating to predict; ensure
the method accepts those parameters and forwards the dataframe to
self.predict(df) (and any necessary kwargs) so overrides/type-checking remain
consistent with other wrappers and BaseSignalModel.

In `@AI/pipelines/weekly_routine.py`:
- Line 30: Remove the unused import of the traceback module: delete the "import
traceback" line in AI/pipelines/weekly_routine.py (the unused symbol is
traceback) so there are no unused imports left; run linters to confirm no other
unused imports remain.

Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/train.yml:
- Around line 27-33: Update the GitHub Actions steps that reference
actions/checkout@v3 and actions/setup-python@v4 to their current supported
releases (e.g., actions/checkout@v4 and actions/setup-python@v5 or the latest
stable major versions); locate the steps that use the strings
"actions/checkout@v3" and "actions/setup-python@v4" in the workflow and replace
them with the updated action versions, then run a workflow lint or dry-run to
verify compatibility.
- Around line 46-51: Replace the fragile echo JSON creation in the "Setup Kaggle
credentials" step with a jq-based safe generator: capture the secrets
($KAGGLE_USERNAME and $KAGGLE_KEY) into environment variables and run something
like jq -n --arg u "$KAGGLE_USERNAME" --arg k "$KAGGLE_KEY"
'{username:$u,key:$k}' > ~/.kaggle/kaggle.json, then chmod 600
~/.kaggle/kaggle.json; this ensures special characters in
KAGGLE_KEY/KAGGLE_USERNAME are properly escaped and produces valid JSON.

---

Nitpick comments:
In @.github/workflows/train.yml:
- Around line 35-44: The Install dependencies step currently runs pip install
with unpinned packages; create a requirements-train.txt listing the exact
package versions (or minimum major versions) for kaggle, pandas, pyarrow,
psycopg2-binary, paramiko, sshtunnel, scp and update the "Install dependencies"
step to pip install -r requirements-train.txt; also enable caching by using the
setup-python/cache or actions/cache keyed on the Python version and
requirements-train.txt (refer to the existing setup-python action and the
"Install dependencies" step) to speed builds and improve reproducibility.

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@AI/modules/signal/models/itransformer/train_kaggle.py`:
- Around line 130-132: The global df.ffill() after merging price_df and macro_df
causes feature leakage across tickers; change the fill strategy to forward-fill
macro columns only within each ticker group and then fill remaining NaNs with 0.
Locate the merged dataframe operations around df = pd.merge(price_df, macro_df,
on="date", how="left"), df = df.sort_values(["ticker",
"date"]).reset_index(drop=True) and replace the df.ffill().fillna(0) step with a
grouped forward-fill for macro columns (e.g., using
df.groupby("ticker")[macro_cols].ffill()) followed by df.fillna(0) to handle
other columns.
- Around line 49-51: The save filenames in train_kaggle.py ("model_name",
"scaler_name", "metadata_name") use the new itransformer_* names but downstream
code (artifact_paths.py, itransformer/wrapper.py, deploy_to_server.py) still
expects the old multi_horizon_* names (e.g., multi_horizon_model.keras,
multi_horizon_scaler.pkl); pick one convention and make it consistent: either
change the keys in train_kaggle.py back to
"model_name":"multi_horizon_model.keras" and
"scaler_name":"multi_horizon_scaler.pkl" (and "metadata_name" if applicable) or
update artifact_paths.py, itransformer/wrapper.py (around the loader at
wrapper.py:339) and deploy_to_server.py to look for itransformer_model.keras /
itransformer_scaler.pkl / itransformer_metadata.json; ensure all references to
model_name/scaler_name/metadata_name across these modules match exactly.

In `@AI/modules/signal/models/TCN/tcn_train_kaggle.py`:
- Line 260: Remove the unnecessary f-string used in the print statement by
replacing the literal print(f"\n>> 완료") with a plain string print("\n>> 완료");
locate the exact statement print(f"\n>> 완료") in tcn_train_kaggle.py (inside the
TCN training/finishing block) and remove the leading f to satisfy Ruff F541
linting.
- Around line 269-270: 현재 parser.add_argument("--end-date",
default="2023-12-31")로 고정돼 있어 자동 재학습이 최신 데이터를 반영하지 못합니다;
parser.add_argument("--end-date", ...)의 기본값을 하드코딩된 날짜에서 동적값으로 바꿔 오늘
날짜(datetime.date.today().isoformat()) 또는 파켓 파일에서 읽어온 최대 날짜(max_date)를 기본값으로
사용하도록 변경하고, argparse 처리 로직에서 --end-date가 주어지지 않았을 때만 이 동적값을 사용하도록 구현하십시오 (예:
compute max_date_from_parquet() 후에 default_end = max(today, max_date)로 설정하거나,
argparse 이후 if args.end_date is None: args.end_date = default_end 방식으로 처리).
- Around line 119-141: The loop that calls add_technical_indicators silently
skips up to 19 failures and then later pd.concat(processed) can raise "No
objects to concatenate" without context; change the feature-calculation loop
(the for ticker in tqdm(...) block) to log each failing ticker and its exception
(include the ticker value and exception text) when add_technical_indicators
raises, keep fail_count but fail faster or at least include a list of
failed_tickers, and after the loop check if processed is empty (or number of
successful tickers is zero) and raise a clear RuntimeError that includes
start_date/end_date, total tickers attempted, fail_count, and the failed_tickers
list before attempting to create full_df; ensure references to processed,
fail_count, failed_tickers, add_technical_indicators, and full_df are updated
accordingly.
- Line 268: The default parquet path set in the
parser.add_argument("--parquet-dir", default=os.environ.get('PARQUET_DIR',
'/kaggle/input/datasets/jihyeongkimm/sisc-ai-trading-dataset')) is invalid for
Kaggle; update the logic so if PARQUET_DIR is not set the code either: 1) uses a
safe default root '/kaggle/input' (so users must provide the specific dataset
slug), or 2) dynamically detects the mounted dataset by scanning /kaggle/input
(e.g., os.walk('/kaggle/input') and picking the matching dataset folder) and
sets the --parquet-dir fallback accordingly; change the code around the
--parquet-dir argument handling to prefer os.environ['PARQUET_DIR'] if present,
otherwise perform the detection or fallback to '/kaggle/input'.
- Around line 155-160: The current split uses "split_date =
dates[split_date_idx]" and then train_df = raw_df[raw_df['date'] <= split_date]
which includes the split_date in the training set and can leave val_df empty;
change the split to make train use dates strictly before the split_date and val
use >= split_date (i.e., use raw_df[raw_df['date'] < split_date] for train_df
and raw_df[raw_df['date'] >= split_date] for val_df) so that the chosen
split_date becomes the first date in the validation set; update references to
dates, split_date_idx, split_date, train_df, and val_df accordingly.

---

Duplicate comments:
In `@AI/scripts/trigger_training.py`:
- Around line 92-98: The code currently inspects result.stdout regardless of
process success; change the logic after the subprocess.run(...) call to first
check result.returncode and immediately return False (and log an error including
returncode/stderr) if it's non-zero, instead of continuing into the 5-minute
retry loop; update the logic around the subprocess.run(...) invocation and the
subsequent use of result/output to bail out on failed `kaggle kernels status`
calls.

---

Nitpick comments:
In `@AI/scripts/trigger_training.py`:
- Around line 65-73: The current success URL and later polling reconstruct the
kernel identifier from KAGGLE_USERNAME and notebook['slug'], which can diverge
from the pushed kernel; instead, after running subprocess.run(...) to push (the
kernels push -p using notebook['notebook_dir']), open and parse the
kernel-metadata.json in notebook['notebook_dir'] to read the canonical kernel
"id" and use that id for both the success message in the block that prints
notebook['name'] and for any subsequent polling logic (the later block around
lines 81-88), replacing usage of KAGGLE_USERNAME/notebook['slug'] with the
metadata id to ensure a single authoritative source for the kernel identifier.

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@AI/modules/signal/models/PatchTST/train.py`:
- Around line 217-219: The current split forces n_val = max(1, int(len(tickers)
* 0.2)) which makes train_tickers empty when len(tickers) == 1 and leads to
divide-by-zero later; change the logic around tickers, n_val and val_tickers so
n_val is at most len(tickers)-1 (e.g. n_val = min(max(1, int(len(tickers) *
0.2)), max(0, len(tickers) - 1))), and if len(tickers) < 2 either raise a clear
error or handle the single-ticker case explicitly (ensure train_tickers is
non-empty before proceeding and adjust downstream averaging logic in the
training loop that uses train_tickers/val_tickers).

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@AI/kaggle_notebooks/itransformer/notebook.ipynb`:
- Around line 32-36: The notebook currently hardcodes the PR branch in the
subprocess.run git clone call, which will break if that branch is removed;
change this to use a branch variable sourced from an environment variable with a
default of "main": import os and set something like branch =
os.getenv("PR_BRANCH", "main"), then replace the hardcoded
'20260326-#343-AI-kaggle-pipeline' token in the argument list passed to
subprocess.run with that branch variable so the clone uses the env override if
provided and "main" otherwise; update the subprocess.run call that builds the
list of git args to reference the branch variable instead of the literal string.

In `@AI/kaggle_notebooks/patchtst/notebook.ipynb`:
- Around line 18-25: 현재 subprocess.run(...)에서 브랜치명이 하드코딩된 git clone 명령(브랜치
'20260326-#343-AI-kaggle-pipeline')으로 인해 브랜치 변경 시 파이프라인이 깨집니다; 변경 방법은
subprocess.run 호출에서 사용되는 브랜치 문자열을 환경변수로 대체하되 기본값을 'main'으로 유지하도록 하세요 (예: BRANCH
= os.environ.get('CLONE_BRANCH', 'main')를 읽어 git clone 인자 리스트의 --branch 값으로 사용),
관련 위치는 subprocess.run 호출과 그 파라미터 문자열을 찾아 수정하고 stdout/stderr 출력
로직(print(result.stdout), print(result.stderr))은 그대로 두세요.

In `@AI/kaggle_notebooks/TCN/notebook.ipynb`:
- Around line 19-23: The notebook currently hardcodes a PR-specific branch in
the subprocess.run git clone call (the branch string
'20260326-#343-AI-kaggle-pipeline'); change this to use a default branch (e.g.,
'main') and allow overriding via an environment variable (read from os.environ,
e.g., GIT_CLONE_BRANCH) before building the arguments passed to subprocess.run
so the clone uses os.environ.get('GIT_CLONE_BRANCH', 'main') instead of the
fixed branch name.

In `@AI/kaggle_notebooks/transformer/notebook.ipynb`:
- Around line 31-35: The git clone call hardcodes a short-lived PR branch;
change the '--branch' argument in the subprocess.run call so it defaults to
'main' and can be overridden via an environment variable (e.g., read
os.environ.get('CLONE_BRANCH', 'main') and insert that into the args list passed
to subprocess.run). Update the list of arguments used in the subprocess.run
invocation (the one containing '--branch', '20260326-#343-AI-kaggle-pipeline')
to use the env-provided branch value instead of the hardcoded PR branch.

In `@AI/modules/signal/models/PatchTST/train_kaggle.py`:
- Around line 239-252: The code doesn't guard against empty sequences from
build_sequences (X_train/X_val), which will crash DataLoader creation or later
loss/avg computations; before creating train_loader/val_loader in
train_kaggle.py (after build_sequences) check that X_train and X_val are
non-empty (e.g., len(X_train) > 0 and len(X_val) > 0 or X_train.shape[0] > 0),
and if either is empty either raise a clear error/log and exit or return early
(skip training) with a descriptive message; ensure the same check is applied at
the other spot mentioned (the block around the later DataLoader creation at
~299-310) so TensorDataset/torch.DataLoader and subsequent average-loss
computations never receive empty tensors.
- Around line 109-113: The code currently only prints missing features (computed
via FEATURE_COLUMNS, available, missing) but must fail fast to avoid later
shape/runtime errors when creating the model with enc_in=len(FEATURE_COLUMNS);
change the check so that if missing is non-empty you raise an explicit error
(e.g., ValueError) or exit with non-zero status including the missing names and
context (full_df columns and expected FEATURE_COLUMNS) so execution stops
immediately instead of proceeding to model creation.
- Around line 47-49: The config currently hardcodes 'end_date' to '2023-12-31',
which excludes all data after 2023; update the code in
AI/modules/signal/models/PatchTST/train_kaggle.py to compute 'end_date'
dynamically (e.g., using today's date) instead of the fixed string so daily runs
include up-to-current data; locate the config entry with the 'end_date' key and
replace the literal with a computed value using the current date (via
datetime.date.today() or datetime.datetime.utcnow().date()) formatted as
'YYYY-MM-DD'.

In `@AI/modules/signal/models/TCN/train_kaggle.py`:
- Around line 161-168: 날짜 기반 분할 후 train_df 또는 val_df가 빈 데이터프레임이 되어
scaler.fit(...)가 실패하는 경계 케이스를 방지하려면 split_date 계산 직후와 train_df/val_df 생성 직후에 유효성
검사를 추가하세요: variables to check are dates, split_date_idx, split_date, train_df,
val_df; if train_df.empty or val_df.empty, adjust split_date by moving
split_date_idx one step (e.g., decrement or increment safely within bounds of
dates) until both sides are non-empty or fall back to a row-based split, and
update train_df/val_df accordingly; if adjustment is impossible (very few unique
dates), raise a clear exception that includes dates and split_date_idx so caller
can handle it; apply the same validation/fix for the other split occurrence
around the scaler.fit(...) call mentioned (lines ~172-174).

In `@AI/modules/signal/models/transformer/train_kaggle.py`:
- Around line 155-159: The train/val split currently builds separate ID maps
causing inconsistent embedding indices; instead build ticker_to_id and
sector_to_id once from the single full_df and pass those same dicts into
train_single_pipeline so both train and val use identical mappings; locate where
tickers/sectors and ticker_to_id/sector_to_id are created and move that logic to
run once (using full_df.unique as shown) and ensure train_single_pipeline
signature/caller uses the shared ticker_to_id and sector_to_id (also update the
places noted around lines 227-228 to accept the shared maps).
- Around line 131-143: The code currently calls full_df = pd.concat(processed)
which will raise a confusing error when processed is empty; modify the loop that
iterates over raw_df['ticker'].unique() to collect failed tickers (e.g.,
failed_tickers list) inside the except block where fail_count is incremented,
and before calling pd.concat(processed) check if processed is empty; if so,
raise a RuntimeError with a clear message that includes fail_count and the
failed_tickers (and optionally the first exception message) instead of letting
pd.concat fail silently.
- Around line 216-223: The current split can leave train empty when tickers has
length 1; add a validation on tickers (the variable computed from
full_df['ticker'].unique()) to ensure at least 2 unique tickers before computing
n_val/train_tickers/val_tickers, and if not, raise a clear ValueError
instructing the user to provide more tickers (or alternatively handle the
single-ticker case by setting n_val=0 and val_tickers empty); update the logic
around n_val, val_tickers, train_tickers, train_df and val_df accordingly so
scaler fitting later (on train_df) never receives an empty DataFrame.

In `@AI/scripts/deploy_to_server.py`:
- Line 101: The remote mkdir call injects model['remote_dir'] directly into
ssh.exec_command which is unsafe for paths containing spaces or shell
metacharacters; change the code that calls ssh.exec_command(f"mkdir -p
{model['remote_dir']}") to either (A) shell-escape or quote the path using
shlex.quote(model['remote_dir']) before interpolating, or (B) avoid the shell
entirely and use the SSH library's SFTP API (e.g., paramiko.SFTPClient.mkdir or
sftp.makedirs) to create the directory; update the location where
ssh.exec_command is invoked and ensure any alternate approach handles parent
dirs and errors robustly.

In `@AI/scripts/trigger_training.py`:
- Around line 35-38: Update the notebook_dir path string to match the actual
lowercase folder name so the existence check succeeds: change the value passed
to os.path.join(project_root, "AI/kaggle_notebooks/patchtst") where the dict for
"name": "PatchTST" / "slug": "patchtst-training" is defined (this is the entry
used later in the folder existence check around Line 61); ensure the path uses
"patchtst" (all lowercase) to match the filesystem.

In `@AI/scripts/upload_to_kaggle.py`:
- Around line 24-27: Before attempting upload, validate that the dataset
metadata file exists and its "id" matches the expected target constructed from
KAGGLE_USERNAME and DATASET_SLUG: read and parse dataset-metadata.json, confirm
the "id" field equals f"{KAGGLE_USERNAME}/{DATASET_SLUG}", and if missing or
mismatched log an error via the same logger and abort (exit non-zero). Update
places that print the upload target (where OUTPUT_DIR, KAGGLE_USERNAME,
DATASET_SLUG are referenced and in the upload routine around lines ~58-67) to
show the actual id from dataset-metadata.json when proceeding. Ensure failures
handle FileNotFoundError/JSON parse errors with clear error messages and do not
attempt to call the Kaggle upload step when validation fails.

---

Duplicate comments:
In `@AI/modules/signal/models/TCN/train_kaggle.py`:
- Around line 287-288: The CLI currently hardcodes
parser.add_argument("--end-date", default="2023-12-31") which prevents automatic
retrain on newest data; change the default to a dynamic value (e.g., compute
today's date at runtime or use None meaning "up to today") and update any
downstream code that reads the --end-date argument (the parser usage sites) to
handle the new sentinel by converting None to datetime.date.today() or
formatting today's date string; ensure you import datetime where needed and
update any tests or invocations that expect the old fixed date.

In `@AI/modules/signal/models/transformer/train_kaggle.py`:
- Around line 82-83: The artifact filenames in the config dict (the 'model_name'
and 'scaler_name' entries in train_kaggle.py) don't follow the deployment
resolver naming convention and may fail during download; update those entries to
match the resolver pattern (e.g., append the required suffix such as "_prod"
before the file extension: change "multi_horizon_model.keras" →
"multi_horizon_model_prod.keras" and "multi_horizon_scaler.pkl" →
"multi_horizon_scaler_prod.pkl"), or alternatively generate names via the
project's artifact-resolver helper (use the existing helper/constant that
produces resolver-compliant names) so both 'model_name' and 'scaler_name' always
follow the deployment naming convention.

In `@AI/pipelines/weekly_routine.py`:
- Around line 169-172: The call to run_weekly_pipeline is ignoring its return
value so failures won't set a non‑zero process exit status; change the main
invocation to capture the function's return (e.g., ret =
run_weekly_pipeline(...)) and propagate it via sys.exit(ret) (or sys.exit(1)
when the function indicates failure) or re-raise/raise SystemExit based on
run_weekly_pipeline's contract; update the invocation site around
run_weekly_pipeline to exit with the proper code instead of discarding the
result.

In `@AI/scripts/deploy_to_server.py`:
- Around line 103-118: The current deploy_model loop always returns True even if
files were missing or transfer failed; modify deploy_model to track failures by
introducing a boolean or list (e.g., had_failures or failed_files) and mark it
when a local file is missing or when scp.put raises/returns an error (wrap
scp.put in try/except and record failures on exceptions or non-success results),
and at the end return False if any failures were recorded, otherwise return
True; reference the for loop iterating model['files'], the
local_path/remote_path handling, and the scp.put call to locate where to add the
failure collection and the final conditional return.
- Line 78: Replace the insecure AutoAddPolicy usage by enforcing host-key
verification: in the SSH client setup (SSHClient instance where
ssh.set_missing_host_key_policy is called) remove paramiko.AutoAddPolicy() and
instead call ssh.load_system_host_keys() or
ssh.load_host_keys(<path_to_injected_known_hosts>) and set
ssh.set_missing_host_key_policy(paramiko.RejectPolicy()); also add clear error
handling in the connection flow (e.g., catch
paramiko.SSHException/BadHostKeyException) to fail the deploy when the host key
is not recognized so the pipeline uses pre-registered known_hosts or injected
secret.

In `@AI/scripts/download_weights.py`:
- Around line 30-51: The download currently trusts returncode==0 and writes
directly into the destination dirs defined in MODELS, which can mask
missing/renamed files; instead download into a temporary directory per model,
verify the presence of the exact expected filenames that deploy_to_server.py
requires, and only atomically replace the model's dst_dir with the temp
directory if verification succeeds. Concretely: add or reference an
expected_files list per model (tied to MODELS["name"] or extend each dict with
"expected_files"), create a tmp dir (tempfile.mkdtemp()) and download the
artifacts there, check that every file in expected_files exists (and non-empty)
in the temp dir, on success remove or backup the existing dst_dir and
move/rename the temp dir to dst_dir (os.replace / shutil.move) to ensure atomic
swap, and if verification fails delete the temp dir and raise/return a non-zero
failure so callers know the download didn't produce the expected outputs (also
update the download function to use these symbols: MODELS, dst_dir, and
expected_files, and make deploy_to_server.py's expected filenames the source of
truth).

In `@AI/scripts/extract_to_parquet.py`:
- Around line 195-202: The SELECT used to build df in extract_to_parquet.py is
missing the features expected by the iTransformer training pipeline—add
correlation_spike and surprise_cpi to the SELECT list so df contains those
columns (the df = read_sql_safe(""" ... """, "macroeconomic_indicators") block);
confirm the column names match the database schema and include them alongside
the existing fields (date, cpi, gdp, etc.) so downstream
AI/modules/signal/models/itransformer/train_kaggle.py receives the expected
features.

---

Nitpick comments:
In `@AI/modules/signal/models/itransformer/train_kaggle.py`:
- Line 170: The loop variable "ticker" in the for statement "for ticker, group
in df.groupby(\"ticker\"):" is unused; rename it to "_ticker" to indicate it's
intentionally ignored (keep "group" as-is) and update any references if they
exist elsewhere in the surrounding function (train_kaggle.py) to avoid
unused-variable warnings and improve readability.

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@AI/modules/signal/models/itransformer/train_kaggle.py`:
- Line 55: The config currently hardcodes "train_end_date" to "2023-12-31",
causing weekly retraining to ignore new data; change the default in the config
dict (the "train_end_date" entry) to None (or remove the fixed date) so the
training pipeline (functions using train_end_date in train_kaggle.py) interprets
None as "use the latest available data"; also update the other hardcoded
occurrences referenced in the diff (the later "train_end_date" usages) to follow
the same pattern and ensure any logic that slices data treats None as no cutoff
(and only applies an external cutoff when an explicit date string is provided).
- Around line 138-146: The macro change calculations rely on row order; sort
macro_df by the "date" column (e.g., macro_df = macro_df.sort_values("date"))
and then deduplicate explicitly (use drop_duplicates(subset="date", keep="last"
or "first") depending on desired policy) before calling diff()/pct_change() so
us10y_chg, dxy_chg, and vix_change_rate are computed on chronologically ordered
data; apply this to the code paths around macro_df, available_macro, and the
subsequent if blocks referencing "us10y", "dxy_close", and "vix_close".
- Around line 223-247: Replace the two uses of layers.Lambda(lambda t:
tf.transpose(...)) with the built-in Permute layer to avoid Lambda serialization
issues: locate the transpose applied to seq_input (currently assigned to x via
layers.Lambda(lambda t: tf.transpose(t, perm=[0, 2, 1]))(seq_input)) and the
final transpose applied to x (layers.Lambda(lambda t: tf.transpose(t, perm=[0,
2, 1]))(x)) and change them to layers.Permute((2,1)) so the model uses Permute
around the Transformer blocks (preserving the [batch, seq, feat] → [batch, feat,
seq] and back transformations) and can be safely saved/loaded without
custom_objects.

In `@AI/modules/signal/models/TCN/train_kaggle.py`:
- Around line 273-281: The _find_kaggle_dataset_path function currently returns
any directory under /kaggle/input that contains any .parquet file; update it to
specifically check for the presence of "price_data.parquet" (instead of any
.parquet) before returning a directory, and fall back to
os.environ.get("PARQUET_DIR", base) if no directory contains price_data.parquet;
keep the same return behavior and error/fallback semantics but use the filename
match to avoid false-positive dataset paths.

---

Duplicate comments:
In `@AI/kaggle_notebooks/TCN/notebook.ipynb`:
- Around line 20-23: 현재 셀에서 subprocess.run으로 git clone을 수행할 때 브랜치
이름('20260326-#343-AI-kaggle-pipeline')을 하드코딩하고 복제 실패 시 처리를 하지 않아 이후 셀들이 원인 불명의
에러를 일으킬 수 있습니다; subprocess.run 호출을 수정해 브랜치와 리포지터리 URL을 하드코딩하지 않고 환경변수 또는 함수/변수
인자로 주입(예: BRANCH_NAME, REPO_URL 또는 notebook parameter)하도록 변경하고,
subprocess.run(..., capture_output=True, text=True) 호출 결과의 returncode를 확인하여 실패하면
stderr/stdout를 로그로 출력하고 예외를 발생시키거나 즉시 중단(exit/raise)하도록 처리하여 후속 실행을 막으세요 (참조:
subprocess.run 호출, 클론 대상 경로 '/kaggle/working/sisc-web').

In `@AI/modules/signal/models/TCN/train_kaggle.py`:
- Line 288: The hardcoded default for the CLI flag in
parser.add_argument("--end-date", default="2023-12-31") causes automated
retraining to skip recent data; change the default to a dynamic value (e.g.,
compute datetime.date.today().isoformat() at runtime or accept an environment
override) or set the argparse default to None and compute end_date from
datetime.date.today() later in the script; update all occurrences of the same
flag (see other parser.add_argument calls around the second mention at lines
~309-310) so the pipeline uses the current execution date by default while still
allowing explicit CLI/ENV overrides.
- Around line 161-174: The split by unique dates can produce an empty train_df
or val_df before scaler.fit; update the logic around split_date_idx/split_date
to validate and adjust the split so both train_df and val_df are non-empty
before scaling: after computing dates and split_date_idx, loop or adjust
split_date_idx (e.g., move +1 or -1) until raw_df[date < split_date] and
raw_df[date >= split_date] both contain rows, or fall back to a safe default
split (e.g., single-date guard or raise a clear error), then proceed to fit
StandardScaler on train_df[FEATURE_COLUMNS] and transform both; ensure you
reference split_date_idx, split_date, train_df, val_df, FEATURE_COLUMNS, and
scaler in the change.

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@AI/modules/signal/models/itransformer/train_kaggle.py`:
- Around line 216-221: The model built by build_model currently only declares a
single Input("sequence_input") while the inference wrapper expects three inputs
([feature_frame, ticker_id, sector_id]); update build_model to expose a matching
input signature by adding inputs for ticker_id and sector_id (e.g., Input layers
for those IDs), incorporate their embeddings or processing into the model
pipeline (concatenate or broadcast/expand as needed with the sequence_input
features), and return a tf.keras.Model that accepts the three tensors so
wrapper.predict([feature_frame, ticker_id, sector_id]) will match the saved
model signature.

---

Duplicate comments:
In `@AI/modules/signal/models/itransformer/train_kaggle.py`:
- Line 55: The hardcoded train_end_date value ("2023-12-31") in train_kaggle.py
causes post-2023 data to be excluded every week; change the default for
train_end_date to None (or to read an optional BACKFILL_TRAIN_END_DATE env var)
so training uses data up to "now" by default, and only apply a fixed cutoff when
an explicit environment variable is provided for backfill runs; update
references that read train_end_date (the variable named train_end_date around
the train configuration/initialization) to handle None as “no cutoff” and
convert an explicit env var to a date when present.
- Around line 223-224: Replace the serialization-unsafe Lambda(tf.transpose)
usage with Keras' built-in Permute layer to avoid load/save custom object
issues: locate the Lambda call that transposes seq_input (layers.Lambda(lambda
t: tf.transpose(t, perm=[0,2,1]))(seq_input)) and replace it with
layers.Permute((2,1)) applied to the input (same replacement for the duplicate
at the other occurrence), ensuring the perm tuple swaps the time and feature
axes (Permute uses dims excluding batch).
- Around line 138-146: The change-rate calculations for us10y/dxy/vix are
order-dependent; before calling diff()/pct_change() ensure macro_df is sorted by
the date column and that date is a proper datetime type: convert the date column
to datetime if needed, run macro_df = macro_df.sort_values("date") (or
sort_values(by=["date"]) and then drop_duplicates("date")/reset_index(drop=True)
so that the subsequent us10y_chg, dxy_chg and vix_change_rate computed from
macro_df["us10y"].diff() and pct_change() use correctly ordered rows.

---

Nitpick comments:
In `@AI/modules/signal/models/itransformer/train_kaggle.py`:
- Line 41: Rename unused loop variables to start with an underscore to silence
Ruff B007: change the second variable named dirs to _dirs in the os.walk loop
(the line containing "for root, dirs, files in os.walk(base):") and likewise
rename any unused ticker variable at the other occurrence (the loop around line
188 where _ticker is suggested) so they become _dirs and _ticker respectively;
this preserves behavior while removing lint warnings.